Skip to content

fix: use spill depth partition bit in new final agg#19626

Merged
dqhl76 merged 7 commits intodatabendlabs:mainfrom
dqhl76:fix-final-aggregate-bug
Apr 7, 2026
Merged

fix: use spill depth partition bit in new final agg#19626
dqhl76 merged 7 commits intodatabendlabs:mainfrom
dqhl76:fix-final-aggregate-bug

Conversation

@dqhl76
Copy link
Copy Markdown
Collaborator

@dqhl76 dqhl76 commented Mar 27, 2026

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

part of #19622

Context: In the final aggregation stage, incoming data is already partitioned by the partial aggregation stage. As a result, when spilling occurs, all data falls into a single bucket.
Issue: Our design intended to halve memory usage during a spill, but a bug prevented this from happening.

Fix: This commit resolves the issue by adjusting the start bit, which correctly splits the data into two parts.

Testing: Verified by regression test test_partition_mask_with_start_bit.

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions Bot added the pr-bugfix this PR patches a bug in codebase label Mar 27, 2026
@dqhl76 dqhl76 marked this pull request as ready for review March 27, 2026 06:12
@dqhl76 dqhl76 requested a review from zhang2014 March 27, 2026 06:13
@dqhl76 dqhl76 marked this pull request as draft March 27, 2026 06:40
@dqhl76 dqhl76 marked this pull request as ready for review March 27, 2026 07:21
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c9cccc35fc

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Revert "cargo fmt"

This reverts commit c9cccc3.
@dqhl76 dqhl76 force-pushed the fix-final-aggregate-bug branch from c9cccc3 to 9ce4bc5 Compare March 27, 2026 07:37
@dqhl76 dqhl76 enabled auto-merge April 7, 2026 06:42
@dqhl76 dqhl76 added this pull request to the merge queue Apr 7, 2026
Merged via the queue into databendlabs:main with commit be229e5 Apr 7, 2026
91 checks passed
@dqhl76 dqhl76 deleted the fix-final-aggregate-bug branch April 7, 2026 07:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-bugfix this PR patches a bug in codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants